A Coordination Service for Distributed Applications

ions. For example, coordination support in distributed shared memory abstractions [AHJ91] typically ensure consistent access to data and may not be suitable for other forms of synchronization (for example, to ensure that two objects A and B are updated atomically). On the other hand, frameworks for transaction management support atomicity but also enforce additional serializability requirements that may not be suitable for some applications. Thus, there is a need for a coordination service that is not tied to any speci c abstraction. Any set of processes or agents in an application (web-based or otherwise) should be able to use this service to coordinate their actions. This service only needs to know the name of the actions to be performed and the allowable sequences of actions. The service interacts with the application via noti cations, permissions and triggers. For example, a set of processes should be able to use this service to coordinate access to a resource in a customized manner (e:g:; round robin) or to ensure that the sequences of actions performed by the processes are synchronized according to some global plan (e:g:; if processes A, B and C perform actions a, b and c respectively, then a global plan may specify that b and c must be performed after a and that both B and C must be ready to do b and c respectively before either action can be performed. Alternatively, we may specify that perform b only if c has not been performed). This service can also allow a designer to integrate separately developed applications and impose inter-application coordination constraints. In this paper, we propose a framework to enable development of collaborative applications in an open system. The framework advocates aspect oriented programming by isolating the following aspects: (a) local computation for the actions, (b) coordination of the actions of di erent participants, (c) transfer of data between the participants and (d) session management. We propose a collaboration abstraction that provides services for each of the following aspects: Coordination Service: This service allows a set of participants in an open system to coordinate the execution of their actions according to a coordination speci cation. The coordination speci cation is given by the creator of the session using a petrinet whose transitions are labeled by actions that the various participants may perform. It is the responsibility of the coordination service to map the requirements to the underlying communication system. Information Exchange Service: This service provides mechanisms for entities to exchange and share data. Although our framework uses channels as the mechanism, it can be extended to include shared objects. This service provides the capability of constructing channels with di erent quality of service parameters such as reliability, jitter control, ow control, etc. Session management service: In our framework, the execution of an application is organized in terms of \sessions" and processes may join/leave sessions. The session management service allows a user to create a session, locate existing sessions and enforce \session rules" at run time. Using session rules, a designer may specify rules regarding joining or leaving a session. Each of these services accommodates the notion of role types and is therefore suitable for open systems. The framework is written in Java and is completely portable. 2 Motivation and Overview Consider the development of a teleteaching application that provides a service to a set of users (students and instructors). In our implementation of this application [SG97], at the instructor's machine, a graphics user interface displays a list of slide numbers, a text frame and image frame. A similar interface is present at the student machines. An instructor may click on a slide number and the corresponding image is displayed on the image frames at both instructor and student machines. The instructor may then speak to explain the slide. A student may click on the ask question button to obtain permission to speak. Alternatively, a student may type in the question in the text frame that is asynchronously transferred to the text frame at the instructor. To support the development of such an application, there are several features to be supported: Since several instructors may want to use this application to teach di erent courses at the same time, the execution has to be organized in terms of sessions, where a session is an instance of an application. Our framework allows a user to create sessions, locate existing sessions and join them. In addition, it allows the creator of the session to enforce restrictions regarding joining and leaving a session. For example, we may disallow more than one instructor in any session and require at least min number of students before the session starts. The session management service provides the capabilities to enforce these rules at run-time. The various activities (such as an instructor sending slides and audio data, a student asking a question) will involve information exchange between the participants. One could provide an abstraction of shared windows for this purpose. Irrespective of whether or not an abstraction of a shared window is provided, multicast channels with transport protocols appropriate for sending image data (large volume transfer perhaps without reliability) and for sending audio data will be needed. The information exchange service in our framework provides the 2 s1 s2 s3 p1 p2 p4 p3 Coordination Service Channel Service Session Management Service servers collaboration objects participants Figure 1: Structure of an Application capability on creating such channels. Higher level abstractions (such as shared windows) can be built by users on top of this infrastructure. Coordination will be required in this application to ensure orderly execution of activities. For example, we may disallow two students to ask a question at the same time and we may disallow a student to ask a question while the instructor is presenting a slide (alternatively, a student may interrupt the instructor). The coordination service provides the capabilities of coordinating the execution of actions by various processes. Essentially, P is a session consisting of processes P1; : : : ; Pn and action(Pi) be the set of actions of Pi. Let local(Pi) de ne a set of constraints on action(Pi) Pi may enforce locally (for example, Pi may specify a particular order in which it may want to execute its action). The coordination service is then provided with a petrinet whose transitions are labeled with actions from [1 i naction(Pi). This petrinet essentially denes global constraints on the set [1 i naction(Pi). The service interacts with the processes by appropriately granting permissions, providing noti cations and triggers so that the actions at di erent processes are executed according to the global constraints. That is, it ensures that all execution sequences of P satisfy local as well as global constraints. For example, if the coordination requirements specify a student may not ask a question during slide presentation, then the coordination service will ensure that the ask question action at a student is not enabled until the end slide event occurs. The main components of our system are the servers and collaboration objects. We assume that a set of servers have been started at a set of machines (see Figure 1). To begin the session, the initiator creates a collaboration object (using the collaboration.java le that we provide) which connects to a nearby server. The initiator then invokes the create session method of this object and provides as parameters three JAVA class les, describing the session management rules, the coordination requirements and the channel speci cations. The server downloads these requirements and a session is created. The designer must also provide a class le for each participant type describing the various actions each participant may perform (for example, in the teleteaching le, two les must be provided, one describing the actions of an instructor and the other for a student). To join a session, a user must download the appropriate participant le. Thus, a student wishing to join the session will create a student object (using the student class le). The student object in turn will create an instance of the collaboration object whose interface is used to access the services of the system. 3 Related Work The notion of work ow has been used to organize a number of distributed systems and several commercial work ow-based systems are available [AAA96, ANRS92]. A work ow consists of a number of activities that have to be ordered according to a speci cation. The activities may include updates to a database or may involve human interactions. Thus, a work ow involves a predetermined sequence of activities to be performed and a work ow engine essentially invokes the activities as per the speci cation. On the other hand, in a collaborative application, the processes determine the actions to be performed and the coordination service must constrain them according to a specication (thus, the coordination service does not invoke activities as in a work ow). The notion of \ oor" has been proposed as an abstraction for coordination in several collaborative systems [DG95]. For example, a oor may be associated with a resource that requires mutually exclusive access. The participating processes will then have to \acquire" the oor before accessing the resource. Token-based algorithms have been proposed in [DG95] wherein one token is introduced for each oor. [FA93] has proposed the notion of \synchronizers" to coordinate multiple distributed objects. A synchronizer can be used to specify constraints on the invocation of methods on various objects. Our framework can be viewed as a generalization of this notion of synchronizers. The Linda model for coordination has been widely used in a number of systems [HKCG91]. In this model, the framework maintains a shared tuple space into which participants may add/delete tuples. This provides for an indirect form of communication (when a participant wants to receive a tuple of a certain type, it simply picks the tuple from the shared space without knowing the producer of the tuple). In this model, both communication and information exchange is performed 3 via the tuple space. A similar event-noti cation service is provided in Corba [Gro95] that allow anonymous communication between producers and consumers of events. Again, in this case, both synchronization and information transfer is achieved via events. Both, the Linda model and event-noti cation service provide producer/consumer synchronization at the base level upon which further capabilities can be built. Our framework separates coordination and information transfer and provides a direct way to specify coordination and allowable sequences of actions (in Linda, one has to code the sequences via the tuple space). Although Corba o ers other specialized services such as a transaction service, it does not provide a coordination service of the kind we have proposed. 4 Speci cation of a distributed application To access the services of our system, we provide a class Collaboration:java whose methods are shown in Figure 2. The create session method is used by a participant to create a session of the application. As a result of this creation, a session id is returned to the creator which is then used to participate in this session. Thus, each process that wishes to participate creates the Collaboration object. A participant may use the poll method of the collaboration class to obtain the ids of the currently active sessions. Once the session identity is obtained, the enroll method is used to parclass Collaboration f int create session(String channel le, String coord le, String session le) int enroll(int session id, String p name, String p type) int leave(int session id, String p name, String p type) Idvector poll() void send(String ch name, Mtype m, String p type) Mtype receive(String ch name, String p type) int checktransition(String action name) void notify(String action name) g Figure 2: Methods of Collaboration Class ticipate in a speci c session. The creator of the session may customize the session by specifying the session management rules, channel specifications and coordination specifications. The participants in the session are then \glued" by the collaboration abstraction according to these speci cations. We now discuss the speci cations in some detail: Session Management Service: There may be several rules governing the participation of processes in an application. The designer must specify these rules in a class that implements the session interface. The session interface consists of two methods, enroll restrictions and leave restrictions. The designer must supply the body of these functions to enforce the restrictions. The designer may use several parameters such as IP addresses and current number of participants of a particular type in the body of the functions and it is the responsibility of the session service to supply the current values of these variables when these methods are invoked. Channel Service: To specify the channels, a designer must provide a Java class consisting a table in which each entry describes a channel. Each entry is of the form < ch; prop1; prop2; : : : ; propn >, where ch is the channel name and propi is a property name. The property name must be a valid name from a set Qos Properties. For example, < ch; reliable; F IFO > is a channel that delivers messages reliably and in order. The collaboration abstraction reads information from this table and creates appropriate channels. To use these channels, the methods send and receive are provided in the Collaboration class as shown in Figure 2. Coordination Service: In general, coordination between participants will require sequencing of actions (an action at one participant has to wait for the occurrence of some actions at other participants), resolving con icts between actions (two or more participants may be ready to perform some actions but only one can be allowed to do so) or synchronizing actions (two participants are blocked until both are ready to perform their respective actions). The coordination service is customized by the application by providing a speci cation of the required coordination. We nd that a petrinet is a simple and compact formalism to specify the coordination requirements. We use a variant of a petrinet that allows us to specify priority and atomicity (due to lack of space, we omit the details of the petrinet). The coordination service only needs to know the names of the actions and not their internal behavior. The service then implements the coordination by creating appropriate control channels and signaling. This petrinet is encoded in a Java class that inherits the base petrinet class. This class consists of two tables, place table and transition table. Each row of place table holds the name of the place and a reference to an object specifying the current number of tokens of each type. Each row of transition table consists of four entries, the transition name, the transition type (explained below), the function guard, and the function to remove and add tokens. Thus, the designer must ll in the appropriate entries in these tables to describe the petrinet. Since a protocol may involve several di erent types of entities, a designer must specify a class le p type:java for each participant type p type whose methods describe the actions that can be performed 4 by that participant type. For example, the instructor le may include methods such as start slide, end slide, and answer question whereas a student class may include listen and ask question methods. The body of each method may only refer to the local variables and operations on the collaboration object C (this may include sending/receiving messages on various channels). The application may enforce certain local restrictions on the order in which a participant may perform the actions. For example, we may require the execution of start slide and end slide must alternate. The global restrictions are enforced by the coordination service. To enforce the global restrictions e ciently, we require that the designer categorize the actions in each participant module into the following categories: Conflicting actions: An action a is a conflicting action if there exists a reachable global state g such as a is enabled in g and there exists action b at another participant such that execution of b may disable a. Thus, the execution of a con icting action must be resolved globally. Triggered actions: An action a is a triggered action if a must await the occurrence of some event at another site and after a is enabled, no action at another site can disable it. Synchronized actions: An action a is a synchronized action if its execution has to be synchronized with the execution of another action b at some other participant. Local actions: A local action a is an action that can be performed at any time. In addition, for the rst three types of actions, the designer must also specify whether or not the action is blocking (an action is in blocking if the participant is committed to execute this action; for such actions, the coordination service must eventually grant permission whereas for a non-blocking action, the service sends a rejection if the action can not be scheduled immediately). In the future, we plan to add time bounds specifying how a process is willing to wait to obtain permission. Each action requires a di erent sequence of interactions between the participant, collaboration object and the servers. For example, in the case of blocking triggered actions, a participant simply signals its intention to perform the actions and then simply waits for a noti cation. On the other hand, for a local action, the participant simply informs the server of the occurrence. Given the actions that various participants may perform and the types of the actions, the class les of the participants are automatically preprocessed to include appropriate statements to interact with the coordination service. For example, if a is a conflicting action then we add the statement C:checktransition(a) at the beginning of the body of a, where C is the collaboration object. If a is a triggered action then we add the statement C:checktransition(a) at the beginning of the body of a and C:notify(a) to end of the body of a. The body of the methods checktransition and notify in the collaboration class contains the necessary communication to interact with the servers. The checktransition method, for example, sends a message request permission to its system server. This system server may in turn communicate with other system servers to acquire the token necessary to re the transition. When it is possible to re the transition, a message is sent back to a collaboration object which results in completion of the checktransition method. Due to lack of space, the details of the algorithms used to implement the coordination service will be given in the full paper. 5 Description of the Server In this section, we describe an architecture of the server. At start up time, a set of system servers are started on an appropriate set of machines. Each system server listens on a well-known port number for potential participants to connect to it. When a Collaboration object is created, it connects to a speci c system server. The services of the abstraction are accessed via the system server. The architecture of the system server is given in Figure 3. The system server Session Table Coordination Spec Table Configurable Channel Module data and control messages from other system servers system servers data to other input buffers buffers output Figure 3: Architecture of the Servermaintains the following components:Coordination Table: The coordination table is a ta-ble of objects, one for each session. The object for asession encapsulates the global petrinet for the session.It stores the current marking of the petrinet and has amethod to determine whether a transition can be exe-cuted. It also includes statements to communicate withother system servers to acquire tokens for the petrinet.Session table: The session table is again a list ofobjects, one for each session. The object encapsulatesthe session rules and maintains information regardingthe number of participants of each type enrolled in thesession. In addition, it also maintains a list of partici-pants of each type currently connected to that speci c5 system server. This list is used to deliver messages ar-riving on the data channels.Configurable Channel module: The system servercreates an instance of the channel module object whenthe system server is created. Subsequently, the servermay request creation of various types of channels. Thechannel module maintains information regarding thevarious channels and the properties associated with thechannels. The channel module has a library of protocolmodules, one for each QoS property. When a channelwith a list of QoS properties is requested, the appropri-ate protocol modules are selected from the library anda protocol stack is con gured. This module is similarin spirit to the one proposed in [vRBF+95]. However,it allows dynamic recon guration of channels while thesession is in progress. The system is described in moredetail in [SN97, Sur98].6 ConclusionWe have proposed a framework for implementation ofdistributed applications. The framework provides soft-ware support for three di erent abstractions: sessionmanagement, coordination and information exchange.We believe that support for these three abstractionsshould be available as part of the basic infrastructurefor distributed applications. The framework has beenimplemented in Java. We are currently evaluating theperformance and utility of the framework in developingdistributed applications.References[AAA96] G. Alonso, D. Agrawal, and A. Abbadi.Process synchronization in work ow man-agement systems. In Proceedings of theSymposium on Parallel and Distributedprocessing, 1996.[AHJ91] M. Ahamad, P. Hutto, and R. John. Im-plementing and programming causal dis-tributed shared memory. In Proceedings ofthe IEEE Int'l Conference on DistributedComputing Systems, pages 274{281, 1991.[ANRS92] M. Ansari, L. Ness, M. Rusinkiewicz,and A. Sheth. Using exible transac-tions to support multisystem telecommu-nication applications. In Proceedings of theInternational Conference on Very LargeData Bases, 1992.[BKTJ92] H.E. Bal, M.F. Kaashoek, A.S. Tanen-baum, and J. Jansen. Replication tech-niques for speeding up parallel applica-tions on distributed systems. Concur-rency: Practice and Experience, 4(5):337{355, 1992.[DG95]Dommel, H. and Garcia-Luna-Aceves, JJ.Design issuse for oor control protocols.In Proceedings of the Multimedia and Net-working, 1995.[FA93]S. Frolund and G. Agha. A lan-guage framework for multi-object coordi-nation. In European Conference on Object-Oriented Programming, 1993.[Gro95] Object Management Group. The commonobject request broker: Architecture andspeci cation, revision 2.0. 1995.[HKCG91] S. Hufer, D. Kaminsky, N. Carriero, andD. Gelernter. Coordination applications inlinda. In Irisa/Inria Conference on Re-search Directions in High Level ParallelProgramming Languages, 1991.[LEK98] J. Lyon, K. Evans, and J. Klein. Transac-tion internet protocol. In RFC 2371, 1998.[SBS93] D. Schmidt, D. Box, and T. Suda. Adap-tive: a dynamically assembled protocoltransformation, integration and evaluationenvironment. Concurrency: Practice andExperience, 5(4), 1993.[SG97]G. Singh and Y. Gu. An object orientedsystem for developing distributed applica-tions. In International Conference on HighPerformance Computing, December 1997.[SN97]G. Singh and M. Narasimhan. Flexi-ble protocol design for distributed appli-cations. In International Symposium onHigh Performance Computing and Appli-cations, July 1997.[SSTZ93] Schmidt, D., Stiller, B., Tantawy, A., andZitterbart, M. Language support for ex-ible application-tailored protocol con gu-ration. In Proceedings of the IEEE Confer-ence on Local Computer Networks, 1993.[Sur98]S. Suryanarayan. A framework for build-ing con gurable channels. Master's thesis,Kansas State University, 1998.[vRBF+95] R. van Renesse, K. Birman, R. Friedman,M. Hayden, and D. Karr. A frameworkfor protocol composition in Horus. In Pro-ceedings of the Symposium on Principlesof Distributed Computing, 1995.6